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Abstract: An asymptotic framework for optimal control of multiclass stochastic processing net¬ 
works, using formal diffusion approximations under suitable temporal and spatial scaling, by 
Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been 
developed by Harrison (1988). This framework has been implemented in many works for con¬ 
structing asymptotically optimal control policies for a broad range of stochastic network models. 
To date all asymptotic optimality results for such networks correspond to settings where the so¬ 
lution of the EWF is a reflected Brownian motion in the positive orthant with normal reflections. 
In this work we consider a well studied stochastic network which is perhaps the simplest example 
of a model with more than one dimensional workload process. In the regime considered here, the 
singular control problem corresponding to the EWF does not have a simple form explicit solution, 
however by considering an associated free boundary problem one can give a representation for an 
optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain 
whose boundary is determined by the solution of the free boundary problem. Using the form of 
the optimal solution we propose a sequence of control policies, given in terms of suitable thresh¬ 
olds, for the scaled stochastic network control problems and prove that this sequence of policies is 
asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires 
a server to idle under certain conditions which are specified in terms of the thresholds determined 
from the free boundary. 
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1. Introduction 

Stochastic processing networks arise commonly in manufacturing, computer and communication sys¬ 
tems. Optimal regulation is a key objective in the design of such networks. Construction and imple¬ 
mentation of an optimal control can in general be challenging due to complex dynamics, lack of simple 
Markovian state descriptors, and high frequency and throughput characteristics. An approach pioneered 
by Harrison [13] is to approximate the control problems for such complex networks, when the system is in 
heavy trajfic, through certain control problems for Brownian motions. These Brownian control problems 
(BCP) are quite non-standard in that the control processes may not even have bounded variation sample 
paths. A key result of Harrison-van Mieghem [16] says that in quite general settings there are equivalent 
workload formulations (EWF) of such BCP which correspond to more tractable control problems. In the 
EWF, controls are bounded variation processes and thus these problems fall within the classical frame¬ 
work of singular stochastic control, although here one has the additional feature of state constraints 
in non-smooth domains (typically the state space is a convex polyhedral cone). Furthermore, in many 
examples the EWF is of much lower dimension than the original BCP, thus providing significant model 
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simplification. In recent years there have been many works that have developed asymptotically optimal 
control policies for a range of network models by analyzing the solutions of the associated BCP and 
EWF [1, 18, 14, 2, 11, 7]. Specifically, these works consider a sequence of control problems, indexed by 
a parameter n, for the underlying network such that as n becomes large the traffic intensity approaches 
criticality. An asymptotically optimal control is a sequence of policies {T*} such that the (scaled) cost 
when T* is used in the n-th network is asymptotically the lowest that is achievable among all such 
control sequences. One simplifying feature of all the models in the above papers is that the associated 
EWF is a one dimensional singular control problem in ]R_|_ with a monotonic cost function. Such control 
problems have a simple solution given through a one dimensional reflected Brownian motion and this 
explicit form plays a key role in the proofs. 

In [4] a well known queuing system with three buffers and two stations (see Figure 1), in heavy traffic, 
for which the associated EWF is a two dimensional singular control problem has been analyzed. This is 
perhaps the simplest non-trivial model with more than one-dimensional workload process. The model, 
referred to in the literature as the crisscross network, has been previously analyzed in [17, 27, 22, 21]. 
The network is of interest in its own right, but its analysis also gives insight for large networks with 
bottleneck sub-systems that have similar features as the criss-cross network. A brief description of the 
network is as follows - there are 3 classes of customers (corresponding to 3 buffers) and 2 servers; for 
k = 1,2, customers of Class k arrive according to a renewal process and receive service at Station 
1; Class 1 customers leave the system once their service is completed; Class 2 customers after being 
processed at Station 1 proceeds to Station 2, where they are re-designated as Class 3 customers and get 
processed at Station 2 after which they leave the system. Precise descriptions of the control problem 
and the cost criterion are given in Sections 2.2 and 3 respectively. The form of an optimal control 
for this network depends on the underlying parameters, in particular on the strictly positive holding 
cost vector ( 01 , 02 , 03 ) and the (asymptotic) service rate vector (/ii,p, 2 , 1 * 3 ), and in general can be quite 
complex. Indeed the paper [22] discusses two distinct regimes where the structure of optimal control 
policies are expected to be quite different. These regimes are as follows. Case I: oi/ri — C 2 /i 2 + 03/12 < 0, 
and Case II: oi/ii — 02/12 + 03/12 > 0. The paper [22] further differentiates Case II into four sub-cases: 
Case IIA: 02/12 — C 3/12 > 0, 02/12 — oi/ii > 0; Case IIB: 02/12 — 03/12 < 0, 02/12 — ci/ii > 0; Case IIC: 
02/12 — 03/12 > 0, 02/12 — Cl/ll < 0; Case IID: 02/12 — 03/12 < 0, 02/12 — ci/ii < 0. 

Case I is the simplest to analyze and it has been shown in [27] that a simple priority policy (server 
1 always gives priority to Buffer 1) is (asymptotically) optimal. Asymptotically optimal control policies 
for Case IIA have been constructed in [22, 4]. In this case using certain monotonicity properties one 
can show that the optimal state process for the singular control problem is a two dimensional reflected 
Brownian motion in with normal reflections. Furthermore, using results of [16] an explicit solution of 
the BCP can be given as well. The proposed policies and the proof of asymptotic optimality in [22] and 
[4] are quite different - the first paper uses quite technical machinery from viscosity solution analysis 
of Hamilton-Jacobi-Bellman(HJB) equations whereas the paper [4] proceeds by constructing a multiple 
threshold policy for which the associated state process closely mimics the solution of the BCP. 

Cases IIB, IIC and IID, to date have remained unsolved. One of the key obstacles in their analysis 
has been that in these regimes the singular control problem does not admit a simple form solution. 
Indeed, although Harrison’s framework has provided asymptotically optimal control policies for a broad 
range of models, all of the available results correspond to settings where the solution of the EWF is a 
reflected Brownian motion in the positive orthant (with normal reflections). In [9] the authors studied 
the singular control problem that corresponds to the EWF for Cases IIB and IIC. Typically, solutions 
of singular control problems are given in terms of an open set O in the state space, referred to as 
the continuation region, such that starting within O no control is applied until the boundary of O is 
reached; if the initial condition is in (O)'^ {action region), an instantaneous control in a pre-specified 
direction is applied to bring the state to dO] and once in O, the state process is constrained in the 
set by suitable reflection at dO. In terms of the associated HJB equation, in O the value function 
satisfies a linear elliptic PDE while in a nonlinear first order PDE is satisfied; the boundary dO 
separating these two regions is referred to as the free boundary for the system of PDE and determining 


imsart-generic ver. 2011/11/15 file: crisscross-6.tex date: February 10, 2015 


/Asymptotically Optimal Control. 3 

this boundary is called a free boundary problem. In [9] it was shown that the value function J* of the 
control problem is up to the boundary and the continuation region for the optimal control is given 
as G = {x G : Xi > 'I'(a; 2 )} where : R+ -A R+ is a Lipschitz non-decreasing function given as 

^'(^ 2 ) = supjuii > 0 : dyj^J*{wi,W2) = 0 } . ( 1 . 1 ) 

One novel feature of this result is that here the principle of smooth fit ideas that have been used in 
previous works [3, 15, 24, 26] are not applicable and in fact C^-regularity of the value functions - a 
crucial ingredient in these works - is not available. The paper [9] constructs an optimal controlled process 
as a reflected Brownian motion in G reflected at dG = diG U d 2 G, where the direction of reflection is 
62 = ( 0 , 1 )' on 92 G = {x G K+ : a: • 62 = 0 } and is ei = ( 1 , 0 )' on diG = {a: S : '^{x 2 ) = a::i}. 
The solution is not altogether explicit since in order to determine 'k one need the value function of the 
control problem, however numerical methods for computing the free boundary 'k are available [19, 20, 8 ]. 

The goal of this work is to use the solution of the free boundary problem from [9] (given by the 
function 4/) to obtain an asymptotically optimal control policy for the corresponding crisscross network 
in regime IIB and IIC. We only study Case IIB here since treatment of the other case is expected to be 
quite similar. Recall that Case IIB corresponds to 62/^2 — C 3 /i 2 < 0, 62/^2 — ci/ri > 0 (note that it implies 
cipi — 62/^2 + C 3 /i 2 > 0). In this case serving Class 1 customers reduces cost of jobs in Buffer 1 at an 
(asymptotic) average rate of cipi. Also serving Class 2 customers reduces cost of jobs in Buffer 2 at an 
(asymptotic) average rate of C 2 H 2 and at the same time increases cost at an (asymptotic) average rate of 
C 3 /X 2 for Buffer 3. The condition cip-i > 62/^2 — C 3 /X 2 implies that overall cost is reduced at a higher rate 
if Server 1 processes Class 1 customers instead of Class 2 customers. The second condition 62^2 < (^ 31^2 
says that it is cheaper to keep jobs in Buffer 2 than in Buffer 3. The third condition C 2 /i 2 > ci/xi 
means the cost from the queues processed by Server 1 is reduced more rapidly if jobs in Buffer 2 are 
processed. The first condition suggests that a priority policy that favors Class 1 customers should be 
used, however the third condition says that the minimization of immediate workload at Server 1 will be 
achieved by processing Class 2 customers. Also, always giving high priority to Class 1 customers may 
lead to an undesirable underutilization of Server 2. Thus an optimal policy needs to suitably balance 
these opposing considerations. Additionally, even if there are jobs in the second queue (but say no jobs 
in Queue 1) it may be preferable for Server 1 to idle since holding costs in Queue 3 are higher than that 
in Queue 2. Thus an optimal control is not expected to be a non-idling policy. In Section 3 we describe 
our proposed policy that suitably takes into account the various complex features of this parameter 
regime. Furthermore (as in [4]) the policy is designed so that the associated state process closely mimics 
the solution of the BCP given in Section 4. The policy we propose will require Server 1 to idle under 
certain circumstances which are specified in terms of a threshold determined from the free boundary 4^. 

For asymptotic optimality we will assume that the inter-arrival and service times have finite moment 
generating functions in a neighborhood of 0 (unlike [4] we do not assume these random variables to be 
Exponential). These conditions allow the use of certain large deviation estimates that are key in the 
proof of asymptotic optimality. Such large deviation techniques for obtaining asymptotically optimal 
control policies for stochastic networks were first introduced by Bell-Williams[l] and later also used 
in [14, 4, 2]. Our main result is Theorem 3.3 which under Assumptions 2.1, 2.2, 2.3 and 3.1 proves 
the asymptotic optimality of the control policy in Definition 3.2 with a suitable choice of threshold 
parameters c, Zo,go- This result treats Case IIB but, as noted earlier. Case IIC can be treated in a 
similar fashion. Treatment of Case IID is a challenging open problem. For this case even the solutions 
of the associated EWF and the BCP are currently unavailable. We have the following conjecture for 
the form of the solution to the EWF, which if resolved will be a key step forward in the construction 
of asymptotically optimal control policies in regime IID. 

Conjecture 1.1. Let B be the two dimensional Brownian motion on a filtered probability space 
{.At}t>o, P) as in Definition 4-2 and consider the EWF where h is as in (4.12) and {p,i, p 2 , tts), (ci, 62 , 63 ) 
are as in Case IID. Then there exist functions tk^ : ]R_|_ -a K+, z = 1,2, that are Lipschitz, strictly 
increasing, and 4'i(a;) -A 00 as x —>■ 00 , such that there is a unique pair of {Ft} adapted continuous 
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processes with values in M+ that solve 

w^it) = Bi{t) + sup [Si(s) - ^imis))]- , 

0<s<t 

W;it)^B 2 it)+ sup [S 2 (s)- 4 ' 2 (W^i*(s))]“ , 

0<s<t 

and Ii{t) = supo<s<j [i3i(s) — 5 'i(W 2 *(s))]~, mt) = supo<s<j [- 82 ( 3 ) — 4 ' 2 (Wi*(s))]~ is an optimal 
control for the EWF. 

Note that Case IIB corresponds to a setting where 'I'l = dt, with 'h as in (1.1), and 4*2 = 0. 

The paper is organized as follows. In Section 2 we present the queuing network considered in this 
work. We also introduce our main assumptions on the arrival and service processes. In Section 3 we 
introduce our scheduling policy and state the main result which gives asymptotic optimality of the policy 
with suitable choices of threshold parameters. Section 4 reviews results from [9], in particular we present 
the solution of the Brownian control problem associated with the network from Section 2. Section 5 
contains the proof of our main result: Theorem 3.3. Key steps in the proof of the theorem are contained 
in Theorem 5.4, Theorem 5.5 and Lemma 5.7, the proofs of which are given in Section 6 . Theorems 5.4 
and 5.5 are the most technical parts of the paper. The statement of these results are in the same vein 
as Theorems 4.8 and 4.9 in [4]. However these latter results assume that the interarrival and service 
times are exponential and the proofs in the general distribution case treated here are substantially more 
technical. In particular, the proof of Theorem 4.9 in [4] relies on sample path large deviation estimates 
for Poisson processes, whereas in the proof of Theorem 5.5, we use fixed time large deviation estimates 
(as in Lemma 6.1) for renewal processes. Finally the appendix summarizes some elementary facts about 
the one dimensional Skorohod map that are appealed to in our proofs. 

The following notation will be used. For a Polish space S, I?([0,oo) : S) will denote the space of 
right continuous functions with left limits (ROLL) from [0, 00 ) to S equipped with the usual Skorohod 
topology. Define = {/ G D([0, 00 ) : M) : /(O) > 0}. All stochastic processes in this work will have 
ROLL sample paths. A stochastic process X with values in S will be regarded as a random variable 
with values in Z?([0,oo) : S). Convergence in distribution of S valued random variables X" to X will 
be denoted as X" X. A sequence X" of processes with sample paths in Z?([0,oo) : S) is said to 
be C-tight if the corresponding sequence of probability laws is relatively compact (in the usual weak 
convergence topology) and any limit point is supported on the space of S valued continuous functions. 


4 

( 1 . 2 ) 


2. Queueing Network Model 
2.1. Network structure 

Consider a sequence of networks indexed by u G N of the form in Figure 1. The nth network consists of 3 
classes of customers (corresponding to 3 buffers) and 2 servers. For fc = 1, 2, customers of Class k arrive 
according to a renewal process with rate and receive service at Station 1. Class 1 customers leave 
the system once their service is completed. Class 2 customers after being served at Station 1 proceed to 
Station 2, where they are re-designated as Class 3 customers and get served at Station 2. The service 
rates for these 3 classes of customers are denoted by = 1,2,3. Within each class, customers are 
processed using the First-Come-First-Served discipline. 

A precise mathematical description is as follows. Let (il,X, P) be a complete probability space; all 
random variables and stochastic processes described in this work are, without loss of generality, defined 
on this common probability space. For fc = 1, 2, and j = 1, 2, 3, let {ufe(i) : I = 1, 2, • • • } be a sequence 
of i.i.d. random variables with mean 1 and standard deviation Ok, and {vj{i) : i = 1,2,- ■ ■} a sequence 
of i.i.d. random variables with mean 1 and standard deviation Cj. In the nth network, for fc = 1, 2 and 
j = 1,2,3, the inter-arrival times {u^{i) : i = 1,2,-••} for Class k customers, and the service times 
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Fig 1. The nth crisscross network. 


{v^{i) : i = 1, 2, • • • } for Class j customers are given by 

v^{i) = -/vjii), 

where G (0,oo) are the arrival and service rates. We further assume that the sequences of inter¬ 

arrival times and service times are mutually independent for each n G N. Define 

i i 

efc(0 = E<W’ = forl = l,2,... , A=l,2, j = 1,2,3. 

i=l i=l 

Then the arrival and service processes can be described as follows: 


A^{t) = snp{l > 0 : ^^(1) < t}, S^{t) = sup{l > 0 : < t}, t > 0, fc = 1,2, j = 1,2,3. 

Thus A^[t) represents the numbers of customers of Class k who have arrived up to time t and S^{t) 
represents the number of customers of Class j who would have finished service up to time t if the 
corresponding server had continuously served Class j customers during time interval [0, t]. We make the 
following assumptions on the arrival and service rates. 

Assumption 2.1. For k = 1,2 and j = 1, 2, 3, there exist \k,tLj G (0, oo) such that lim„_>oo = Xt, 

Ihaiyi—J-Qo /ij - fj^j. 


The following is our main heavy traffic assumption. 

Assumption 2.2. The following relations hold for the arrival and service rate parameters: 

Ai A2 _ ^2 _ . 

Ml M2 ’ M3 

and there exist 6^ S K, z = 1, 2, 3, such that 

— hi, z = 1, 2, 


lim 

"->00 pLi 


lim • 

n—^oo 


mi ^ - 1 

.Ms 


= bs . 


( 2 . 1 ) 


( 2 . 2 ) 


Condition (2.1) says the traffic intensities at both stations converge to 1 as n —>■ 00 , while the 
convergence rates of the traffic intensities are characterized in (2.2). 

Finally, we make the following assumption on the logarithmic moment generating functions for inter¬ 
arrival and service times that enables certain large deviation estimates for the renewal processes A!f, Sf 
(see Lemma 6.1). 
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Assumption 2,3. There is a non-empty open neighborhood O o/O € M such that for all I G O, 

Aa,fe = < oo, As,j <oo, k = 1,2, j = 1,2,3. 

2.2. Scheduling control 

Scheduling control for the nth network is described by a vector-valued allocation process 

T"(t) = (rr(<),T”(t),T3”(t))', t>0, 

where for j = 1,2,3, T”(t) represents the cumulative amount of service time devoted to Class j cus¬ 
tomers in the time interval [0,t]. The idle time processes at two servers are defined as follows: 

For simplicity, we assume the system is initially empty. Then the queue-length processes corresponding 
to the three types of customers can be described as follows. For t > 0, 

Qnt) = A^t) - s^irrm * = 1 , 2 , g^(t) = - s^{T^{t)). (2.3) 

We write A” = A" = (A",A 2 )'. The quantities g^,X,g are defined similarly. 

In order to precisely formulate the family of admissible control policies, we need to introduce the 
notion of multi-parameter filtrations and stopping times (cf. Section 2.8 of [12]). Define for a = (oi, 02 )' S 
N2,6= (&i,62,&3)' GN3, 

Q'^{a,h) = cr |<(di),i^”(6j) : di < ai,lj < bj, i = l,2,j = 1,2,3|. 

Then 6) : a G N^,6 G is a multiparameter filtration with the following (partial) ordering 

(d, b) < (a, b) if and only if d^ < a^, bj < bj, i = 1, 2, j = 1, 2, 3. 

A {t/"(a, b) : a € N^, b € N^} multiparameter stopping time is a random variable T which takes values 
in N®, where N = N U { 00 }, such that 

{T = (a, b)} G a"(a, b), for all a G N^, 6 G N^. 

The cr-field associated with such a stopping time is given by 

gii = {BGT:Bn{T={a, b)} G g^{a, b), for all a G N^, 6 G N^} . 

The scheduling control process {T"(t)} is required to satisfy the following conditions. 

(i) For j = 1,2,3 and n G N, T" is a continuous non-decreasing process with T”(0) = 0. 

(ii) For k = 1,2 and n G N, is a continuous non-decreasing process with /^(O) = 0. 

(iii) For j = 1,2,3, t > 0, and n G N, Q^{t) > 0. 

(iv) Define for each t > 0 and n G N, a N® valued random variable 

a^{t) = (AUnt) + 1,1 = 1,2- S^{T^{nt)) + l,j = 1, 2, 3) . 

Then, for each t > 0, o-q (0 is a b) : a G N^, 6 G N^} stopping time. 

Define the filtration {t/"(t) : t > 0} as (t) = Then I'^{nt) is t/"(t) measurable for every 

t > 0. “ 

From (i) and (ii), we see that for all j = 1, 2, 3, 

Tfi is uniformly (in n) Lipschitz continuous with Lipschitz constant bounded by 1. (2-4) 

Although condition (iv) above appears somewhat technical, it is a natural non-anticipativity property 
and Theorem 5.4 of [5] shows that the condition is satisfied for a very broad family of control processes. 

Any process T" satisfying the above conditions will be referred to as an admissible control policy for 
the nth network. 
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Now we define fiuid-scaled and diffusion-scaled processes corresponding to the processes described above. 
For each n € N, define for t > 0, T'"(t) = n“^T”(nt). Processes , A”, , Q" are defined similarly. 
These will occasionally be referred to as fluid scaled processes. 

We also defined diffusion-scaled processes: 

i"(t) = - nA”<), 5”(t) = - np'^t), 

f”{t) = i^{t) = Q”(t) = n-^/^Q^{nt). 


We next define the workload process W” = {(kF"(t), W^(t))' : t > 0}, which measures the amount of 
service needed for customers that are in the system at time t. More precisely, for t > 0, define 


W^{t) = 


Qiit) , Qm 


, w^it) = 


Q^{t) , Q^{t) 


Ml M 2 Ms Ms 

The fluid and diffusion scaled workload processes are defined as follows: For t >0, 


(2.5) 


3. Main Result 


We consider an expected infinite horizon discounted cost, associated with an admissible control T" and 
the corresponding normalized queue-length process Q", given as follows: 


j"(r") = E 



Q^{t)dt , 


(3.1) 


where 7 G ( 0 , 00 ) is the discount factor and c = ( 01 , 02 , 03 )' is a strictly positive vector of holding 
costs. The aim is to find a sequence {T”} of scheduling control policies which is asymptotically optimal, 
namely it satisfies 

lim J"(T”) = infliminf J”(f”") , 

n—^oo n—^00 


where the infimum is over the set of all admissible control policy sequences {T"}„gj!j. We make the 
following assumption on the service rates and the holding cost parameters. 

Assumption 3.1. 02/^2 — 03/^2 < 0, 02^2 — Oi^i > 0. 

As noted in the Introduction, this parameter regime is the Case IIB considered in [22] and an optimal 
policy needs to suitably balance several considerations such as, overall cost is reduced at a higher rate 
if Server 1 processes Class 1 customers instead of Class 2 customers; it is cheaper to keep jobs in Buffer 
2 than in Buffer 3; the cost of jobs in the queues processed by Server 1 is reduced more rapidly if jobs 
in Buffer 2 are processed; and that it is undesirable to have Server 2 idle when there is work in Buffer 
2 . 

We now describe our proposed policy which suitably takes into account the various complex features 
of this parameter regime. The policy is motivated by the form of the solution of the Brownian control 
problem associated with this control problem. This point will be explained further in Section 4 (see 
comments below Corollary 4.6) where the Brownian control problem is studied in detail. 

Fix c,lo G ( 1 , 00 ) and go G (0,oo). Define L" = [/olognj and C" = [colognj, where cq = cIq. Since 
we are only interested in asymptotic optimality, we will assume without any loss of generality, that 
n > fi, where n is such that for all n > fi, C" — L” — 1 > 1 and ^(C" — L” -I- 2) > 1. The control 
policy will be specified in terms of the free boundary {(a:i,a: 2 ) : xi = 4 >(a: 2 )} where is as specified in 
(1.1) with J* as in (4.23). 


imsart-generic ver. 2011/11/15 file: crisscross-6.tex date: February 10, 2015 






/Asymptotically Optimal Control. 8 

Definition 3.2 (Control policy). The policy is as follows. Server 2 processes jobs from Buffer 3 when¬ 
ever the buffer is nonempty. The sequencing policy for Server 1 is as follows. At time s € [0, oo), 


serve Class 1 customers (provided the queue is non-empty) if either Q0{s) > C" — 1 or 
Q^(s) = 0, 

serve Class 2 customers if Qf,{s) < C" — 1 and Q^is) > 0; 

serve Class 1 customers if either Qf{s) > ^(C" — L” + 2) or (Q^is) = 0 and Qf{s) > 0), 

serve Class2 customers ifQfis) < L"+2), Qlfis) > 0 andWf{s)—y/n^{Wf{s)/./n) 

> go, 

idle Server 1 if Qf{s) < ^{C" — L" + 2), Q^is) > 0 and Wf{s) — •v/n'l'(VC^(s)/i/n) < go- 

Remark 3.1. The free boundary 'h is given in terms of the value function J* of the Workload Control 
Problem in Section 4. Although this function does not have a closed form expression, there are well 
developed numerical methods for solving such free boundary problems (see [20, 23, 19, 8 ]). 

Remark 3.2. We note that the proposed control policy addresses many of the complex features of 
the parameter regime. In particular the policy says that if Queue 1 is sufficiently large then it receives 
priority, unless there is a risk of Server 2 idling despite there being jobs in Queue 2. Furthermore, Server 
1 idles when Queue 1 is (asymptotically) negligible and there is enough work in Queue 3, so as to keep 
jobs in the buffer with a lower holding cost. This latter property is enforced by the last line in the 
control policy that involves the free boundary function 4*. 

Remark 3.3. From Theorem 5.4 of [5] it follows that the control policy in Definition 3.2 is an admissible 
control policy in the sense of Section 2.2. An explicit representation for the policy in terms of a vector 
allocation process T" can be given in a manner analogous to Remark 3.8 of [4]. 

The following is our main result, which gives the asymptotic optimality of the policy proposed in 
Definition 3.2. The limit of the cost under the proposed policy is characterized in Theorem 5.2 in terms 
of the solution of a Brownian control problem. 

Theorem 3.3. Let go £ (0,c»). There exist c,l £ (l,c») such that the sequence of scheduling controls 
{T"} defined in Definition 3.2 with threshold parameters c,lo,go, with Iq > I, satisfies 

lim J”(r") = infliminf J”('f"), 


where the infimum is taken over all admissible control policy sequences {T"}. 


Remark 3.4. The parameters c and I in Theorem 3.3 can be chosen as follows. 

For i = l,2,3,j = 1,2, and Z = 1,2, define and 0“’-^ as 0; in Lemma 6.1 with A* replaced by 
A* j and A*_^, respectively, where A*_j and A*_^ are the Legendre-Fenchel transforms of As,i and Aaj, 
respectively. Now let 64 be as in Theorem 5.4. From (6.20) one can see that, 64 can be chosen as follows: 


04 = I inf 




-j min|Ai0“’^(Ai,e), mO"/{p-i, e), poe/(2/ri)| , 


where e £ (0, (^1 — Ai)/4) and Pq £ O. Let c = 1 + ^. 

Next let 74 be as in Theorem 5.5. From (6.45) we see that 74 can be taken to be 


74 = min |dA26i’^(A2, ei)/A, d(0 + l)(/ii - 2ei)02’*(pi, eQ/A, d 6 g,iQf\g,i,e 4 )/K, dpo(6'+ l)ei/(2piA), 
Pi0)’*(Aii,ei)/(4M3), A2e)‘’^(A2,ei)/(4/43), (pi - 2ei)e2’‘(pi, £i)/(4p3), 

(A 2 — 2ei)02’^(A2, £i)/(4/i3), poei/{ 8 fi 3 fii), poei/(8)43X2), i = 2, 3} , 
where ei,d,K,9 are as in (6.26). Finally, let I > max{l, 
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4. Brownian Control Problem and Equivalent Workload Formulation 

We now introduce the Brownian control problem associated with the control problem from Section 3. 
Roughly speaking, the BCP is obtained by taking a formal limit of the sequence of queueing control 
problems. Using the scaling defined in Section 2.3, we have, from (2.3), for a given sequence of admissible 
control policies {T"}, and for all t > 0, 


QUt) = AUt) - S-{T-{t)) + - n2fr{t)) 

'X? A. 




= A-{t) - Snfnt)) + 

\f^i Mi 

Qm = 

= S^(T^it)) - S^if^it)) + ( Ait - f^{t) ) + - T3"(t)). 


For t > 0, let 




X”(t) = i-(f) - S^{fnt)) + 




M” 


X^{t) = 


_K 

, A 2 


t, i = l,2, 


and 


= fntij , * = 1 , 2 , 

From (4.1) - (4.3), we get the following relationships: 

Q7{t) = xnt) + * = 1 , 2 , g-(t) = X-{t) - n-Y-it) + tt^Y,^{t). 

Define for t > 0, 

VMi M 2 / 


(4.1) 

(4.2) 

(4.3) 

(4.4) 

(4.5) 


Write At" = (W”, X^j It can be argued (see for example Lemma 3.3 in [5]) that, under ‘reasonable’ 
control policies, T" => T* and consequently, using functional central limit theorem for renewal processes, 
under such policies 

X" ^ X, (4.6) 

where At is a three-dimensional Brownian motion that starts from the origin and has drift (/ii&i, fJ- 2 b 2 , fJ-sbs- 
H 2 b 2 ) and covariance matrix 


Ai -|- <^3 Ai 0 

0 tT|A2 -I- <ri A 2 


0 

-cfA 


0 


-q"A 2 


C 2 A 2 + <riM3 


Also, 


(4.7) 


lYit) = Y^{t) + Y^{t), and /^(t) = F 3 "(t), t > 0, 
which are nondecreasing processes starting from 0. 

Thus taking a formal limit as n —>■ 00 in (4.4), we arrive at the following BCP. 

Definition 4.1. Let X be a three dimensional Brownian motion as in (4.6), given on some filtered 
probability space (17, P). The Brownian control problem is to find anM.^-valued {iFt}-adapted 

stochastic process Y = {Yi,Y 2 ,Yfi)', which minimizes 


E 


e • Q{t)dt 
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subject to the following conditions. For all t > 0, 

0 < Q^{t) = X,{t) + i = 1,2, 0 < Qa W = X^{t) + (4-8) 


and 


Ii =Yi+ F 2 , I 2 = Y 3 are non-decreasing, and /i(0) = 0, i = 1,2. (4-9) 


We will refer to any {iFt}-adapted process Y = (YijYzjYs)' satisfying (4.8) and (4.9) as an admissible 
control for the BCP and an admissible control that achieves the minimum cost as an optimal control 
for the BCP. 


We now introduce an equivalent workload formulation of the above BCP that makes use of a certain 
static deterministic linear programming (LP) problem. Recall the workload process defined in (2.5). 
Using (4.4) and (4.7), we have for t > 0, 


Wf-{t) = 
W^{t) = 


Q^{t) , Q^{t) 1 "(<) , X^it) 


om 


QW) 

lYf 


xm 


Fix wi,W 2 S [0,oo). Consider the LP problem defined as: 


1^2 

xm 


■int), 


+12 it). 


(4.10) 


minimize 5 i,, 2 ,q 3 ciqi + 0292 + caga 

, . , , 9i , 92 92 , 93 . r, 

subject to-1- =wi, -1-= ^2, 9i, 92, 93 > 0. 


(4.11) 


Ml M2 M3 M3 

A straightforward calculation using the fact that ci/ii — C 2 M 2 + caM 2 > 0 shows that the value of the 
LP is 


h{wi,W 2 ) 


(ciMi)iCi + ^(c 2 M 2 - CiMi)w 2 , when p.^W 2 < M 2 W 1 , 
(C 2 M 2 - caM 2 )wi + (c3M3)w 2, when pL3W2 > P 2 W 1 , 


and the optimal solution is 


* Ml / \ 

9 i = —(M2W1 - M3W2), 
M2 

9 r = o, 


q*=fi 3 W 2 , 93=0, if M 3 i «2 < M2W1, 

9 *=^2lCl, 93 = M3'W2 - M 2 l«l, if M3l«2 > M 2 Wl- 


(4.12) 


(4.13) 


Using (4.12), and taking a formal limit in (4.10), we arrive at the following control problem, which is 
usually referred to as the equivalent workload formulation (for the BCP in Definition 4.1). 

Definition 4.2. Let X and {f2,P, {J^t}t>o, P) be as in Definition 4-L The equivalent workload formu¬ 
lation (EWF) is to find an M.'^-valued {Pt}-adapted stochastic process I = ili,^)', which minimizes 


E 


(f 


-^^h{W{t))dt 


(4.14) 


subject to the following conditions. For all t >11 

0 < Wi{t) = Blit) + hit), 0 < W2it) = B2it) + hit), 

h, h are nondecreasing, and /i(0) = / 2 ( 0 ) =0, 


(4.15) 


where 


Blit) 


Ml M2 ’ M3 M3 


(4.16) 


We will refer to any {Pt}-adapted process I = ih,h)' satisfying (4.15) as an admissible control for the 
FWF and an admissible control that achieves the minimum cost as an optimal control for the FWF. 
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The BCP and EWF of the above form were first introduced by Harrison in [13], and they have been 
used extensively in the study of optimal scheduling for multiclass queuing networks in heavy traffic (see 
[16, 1, 2, 14, 18, 4, 11, 5, 6, 7]). In particular, the BCP and EWF introduced here are identical to those 
in [4]. The following lemma says that in order to solve the BCP it suffices to solve the associated EWF. 
The proof of this lemma is straightforward from (4.12) and (4.13) and we refer the reader to Section 
3.1 of [4] for details. 

Lemma 4.3. Suppose I* is an optimal control of the EWF. Denote by W* the corresponding optimal 
workload (which is defined by (4.15) with I replaced by I*). Define when H 3 Wf{t) < fi 2 W({t), 

Yf{t) = + mt) - ^i*{t), Yf{t) = ^ + ^i;it), Y*{t) = i;it), (4.i7) 

M2 M2 M2 

and when p, 3 Wf{t) > fi 2 W((t), 

Yf{t) = f;(t) = ^ + ipt), Yf{t) = i;{t). (4.i8) 

Ml Ml 

Then Y* is an optimal control of the BCP. 

We note that if Y* is an optimal solution of the BCP, then the corresponding optimal queue length 
is 

Qlit) = ^(M2bFi*(t) - ti3Wf{t)), Q*{t) = M3VF;W, QUt) = (4-19) 

M2 

if M3^2*(^) < T 2 Wf{t), and 

Qm = 0, QUt) = ki2W(it), Q*{t) = pisWfit) - ^l2Wfit), (4.20) 


if fisWfft) > fj, 2 Wf{t). 

Although the BCP and EWF here are the same as those in [4], the solutions to these problems are 
much less straightforward than in [4]. The latter paper considers the Case IIA, where, in particular, 
C 2 M 2 — C 3 M 2 > 0 and C 2 M 2 — ciMi > 0 . In their setting, h is a non-decreasing function of both its 
arguments and consequently, the solution of the EWF is trivial, in that it is given by the solution of 
the one-dimensional Skorohod problem (see Appendix A for the definition and properties of Skorohod 
problem). However for the regime considered here, h no longer has the above monotonicity property 
and thus a simple closed form solution of the EWF or BCP is not available. The BCP in Cases HB 
and HC has been investigated in [9] where using certain optimal stopping problems a solution of the 
EWF has been provided in terms of the free boundary associated with the control problem. Below we 
summarize some key results from [9] that will be needed here. 


Consider the workload control problem in Definition 4.2 corresponding to an arbitrary initial condi¬ 
tion. More precisely, letting B and (17, J", {J^t}t>o, P) be as in Definition 4.2, and w = {wi,W 2 y S 
the control problem is to find an R^-valued {J^t}-adapted non-decreasing process /“ = (ifyifO' to 
minimize 


E 


e-^^h{W'"{t))dt 


(4.21) 


subject to the following conditions: For t >0, 


0 < wfiit) = wi + Blit) + iyit), 
0 < it) = W2 + B2it) + iy it). 


We refer to 
function as: 


ii( 


, lyy as an admissible control for the initial condition w. Define the optimal value 


J*(w) = infE(^^ e-T'*h(W’' 


it))dt 


w € 




(4.23) 


imsart-generic ver. 2011/11/15 file: crisscross-6.tex date: February 10, 2015 






/Asymptotically Optimal Control. 12 

where the infimum is taken over all admissible controls for the initial condition w. 

From Theorem 3.1. of [9], J* is a function on Now define for W 2 € M+, 'h : K_|_ —)> R+ as in 
(1.1). The following result is taken from [9]. 

Lemma 4.4 (Lemma 5.1. of [9]). The function 4* has the following properties. 

(i) For all W2 > 0, 0 < '^>{102) < ^'^2- 

(ii) 'h is non-decreasing and Lipschitz continuous with Lipschitz constant bounded by —. 

(iii) lim 4 '(r(; 2 ) = 00 . 

W2—>-00 

The following result from [9] gives an optimal solution of the workload control problem in Definition 

4.2. 

Theorem 4.5 (Theorem 5.2. of [9]). For t >0, define 

Wf{t) = B,{t) + sup [i?i(s) - 4'(1 T;(s))]- , 

0<s<t 

4.24 

Wf{t) = B 2 {t)-\- sup [B 2 (s)] , 

0<s<t 


where for z £ R, 0 = — min{0, z}. Then the minimum value of (4.14) over all admissible controls is 

given as 


J*(0) =E 


e-^*h{W*it))dt 


(4.25) 


Thus 

Ii{t) = sup [Bi(s) - 4'(1T2 *(s))]” , /^t) = sup [B 2 (s)]“- 

0<s<t 0<s<t 


is an optimal control for the EWF. 


(4.26) 


Consider the one dimensional Skorohod map T : T>i —^ ^([Oj 00 ) : IR+) defined as 


r(/)(t) = fit) + sup if is)) , f eVi, t>0. 

0<s<t 


Then for t > 0, 

Wfit) = TiB 2 )it), Wfit) = r(i?i - 4/(lT*))(f) + 'fiWfit)). (4.27) 

Recall G = {x £ : Xi > 4 '(a; 2 )}. Clearly, 

W*it) G G for all t > 0. (4.28) 

Roughly speaking, the process W* behaves like the Brownian motion (i?i, ^ 2 )^ in the interior of G and 
it is reflected on the boundary of G, where the directions of reflection on {a; G R^ : x • 62 = 0} and 
{x G R^ : Xi = 4 '(x 2 )} are 62 and ei, respectively. 

Combining Theorem 4.5 with Lemma 4.3 we have the following corollary. 

Corollary 4.6. Let W* = iWf^Wf)' and I* = ili,!^)' be as in Theorem 4-5. Then Y* defined by 
(4.17) - (4.18) (replacing I* there with I*) is an optimal control for the BCP and Q* defined by (4.19) 
- (4.20) (replacing W* there by W*J is the corresponding optimally controlled state process. 

The solution to the BCP given above suggests the following control policy for the n-th network. 
Note that the set {Q^it) < ^Qlit)} equals {lasWfft) < fj, 2 W(it)}. Since on this set Q^ft) = 0, a 

good policy for the n-th network should keep Q^it) close to 0 when Qi(it) < ^Qfit). Similarly, when 
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Qsit) > the policy should ensure that Qi{t) is close to 0. This motivates the thresholds 

L" = [IglognJ and C" = [colognj introduced above Definition 3.2. When Q^{t) — < L", 

under the policy in Definition 3.2, Server 1 processes Class 2 customers (leading to an increase in Q'^{t)) 
only when Q^{t) < C" —1. Thus C" can be interpreted as the level of ‘safety stock’ that prevents idleness 
of Server 2. Since in the diffusion scaling the safety stock levels approach 0 (i.e. C"/ ./n —>■ 0 as n — >■ oo) 
the policy ensures that Qa is close to 0 in this regime. Similarly when Q^it) — > T”, Server 1 

processes Class 1 jobs as soon as Q"(t) > — L" + 2), ensuring that in this regime Qi remains 

close to 0. We refer the reader to Corollary 5.6 for a convergence result that makes these statements 
precise. Finally, (4.28) suggests that under a near optimal policy the condition lT”(t) £ G for all t 
should be satisfied approximately for large n. As shown in Lemma 5.7, the policy in Definition 3.2 
satisfies this property. The proof of this lemma relies on the key idleness property formulated in the 
last line of Definition 3.2. 


5. Proof of asymptotic optimality 

In this section we prove Theorem 3.3 which gives the asymptotic optimality of the policy proposed in 
Definition 3.2 for a suitable choice of threshold parameters gQ,c and Iq. We begin with the following 
result which is an immediate consequence of Theorem 3.1 of [5]. 

Theorem 5.1. Let for n>l, T" be an admissible control policy for the n-th network. Then with J*(0) 
as in (4.25), we have, hminf„^oo J"{T^) > J*(0). 

The above theorem says that the optimal cost of the BCP is a lower bound for the asymptotic cost 
for any sequence of admissible control policies. Thus it suffices to show that the sequence of policies in 
Definition 3.2 (with a suitable choice of threshold parameters) asymptotically achieves the optimal cost 
of the BCP. This is done in the theorem below. 

Theorem 5.2. There exist c,l £ (l,oo) such that for any go £ (0,oo) and Iq £ [?, oo), the sequence of 
control policies {T"} in Definition 3.2 with threshold parameters c,lo,go, satisfies the following: 

(i) (W”, r) => (W*,r) as n^oo, 

(ii) J"(T”) ^ J*(0) as n^oo, 

where (IT",/") are defined as in Section 2 using the above sequence of control policies, and IT*,/*, 
J*(0) are as in (4.24), (4.26), and (4.25) respectively. 

Proof of Theorem 3.3 is immediate from Theorems 5.1 and 5.2: 

Proof of Theorem 3.3. From Theorem 5.1 

infliminf J"(f") > J*(0) 

n—¥oo 


where the infimum is taken over all admissible control sequences {T"}. Also, with {T"} as in Theorem 
5.2, 

infliminf J"(f") < liminf i"(r") = J*(0). 


Combining the above two inequalities, we have inf liminf„_>oo j"(T") = J*(0) = lim„_>oo j"(T"). □ 


Rest of this section is devoted to the proof of Theorem 5.2. The proof relies on three technical results: 
Theorem 5.4, Theorem 5.5 and Lemma 5.7, the proofs of which are postponed to Section 6. Throughout 
this section {T"} will denote the sequence of control policies in Definition 3.2 with some choice of 
threshold parameters. We begin with the following lemma from [4]. 
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Lemma 5.3 (Lemma 4.7 of [4]). Let {fn} and {gn} be sequences of functions in D([0,oo) : R), and 
let f and g be continuous functions from [0,oo) to K, such that fn^f,gn^g in L*([0, oo) : R) as 
n —>■ oo. Suppose that /[q oo) = 0- be a sequence of non-negative real numbers 

converging to 0. Then for all T > 0, the following hold: 


f e [ e '^*f{t)l{g(t)>o}dt 

Jo Jo 


as n —>■ oo , 


e fn{t)i{g^(t)<eri}dt 


e fit)l{g(t)<o}dt 


as n —>■ oo . 


Given c G (1, oo), let 


k(c) 


= max 


4^ 

T 2 ’ 


4 c 
c- r c-1 



For n G N, K > k(c), and t > 0, define the events: 


(5.1) 


A(n, t) 



^QUt)<L^ 

ri 


c : ■ f An: M k(C'"-L" + 1) 

Snin^tj s sup Q^{s)l_^(^n,ns) ^ t — 

Lo<s<t vn 


, /in/ M k(C”-L" + 1 ) 

U<^ sup Qiis)lA{n,ns)-= > - 1= - 

[o<s<t vn 


(5.2) 


Proofs of the following two results are given in Section 6. 


Theorem 5.4. There exist Qi G (0, oo), i = 1,2,3,4 and no G N such that for the sequence {T"} with 
threshold parameters Iq > l,c > 1, go > 0 and with £K,(n,t) defined as in (5.2) 


f’{£^{n,t)) < 6 li(nt+ l) 2 e-®="* + 6 » 3 (nt+ 


(5.3) 


whenever k,> k{c), n > no and nt > 2. 

Theorem 5.5. There exist ni G N, e G (0,1), 7 i = 7 i(c) G (0, oo), i = 1, 2, 3,4, and d = d(c) G (0, oo), 
such that for the sequence {T"} of control policies with threshold parameters c > 1 and arbitrary lo > 1, 
go > 0 , 


P 



lBAn,s)di^is) ^ 0 


< 71 (nt + 1)^6-^'="* + Joint + l 


whenever n > ni and nt > 2/e, where 


Bdin,t) 


g"(t) > 


dlo log n 1 

^7 ^/■ 


(5.4) 


An immediate consequence of the above two theorems is the following. 

Corollary 5.6. Let 9^ be as in Theorem 5.4- Let c = 1 + and let ji G (0, oo), i = 1, 2, 3,4 and d be 
as in Theorem 5.5. Choose I G (1, oo) to be large enough so that J 4 I > 3. Fix t > 0. Then for all lo > I, 
go > 0 and sequence {T"} of control policies with threshold parameters cjo^go the probabilities in (5.3) 
and (5.4) tend to 0 as n —>■ 00 for all k > /v(c). In particular, as n ^ 00 , 


Qli-)^A(n,n.)’= ^ 0 , Q 3 (-)lyl(n.n-) ^ 0 , 


Fn,s)dl2is) ^ 0 . 
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For the rest of this section we fix threshold parameters c^lo^go and constants d, k, as in Corollary 
5.6. We will suppress k and d in the notation for Bd(n, s), s). We next provide a lower bound for 

W"(t) — ^/n'^{W 2 {t)/^/n),t > 0, which will ensure that W"(t) € G asymptotically. The proof is given 
in Section 6. 


Lemma 5.7. There exist Ci,C 2 S (0,oo) such that for t>0, we have 

wf{t) - 


1 

> — 
- 


1 - 






^2 


1 - 


T3.T2 




{2Q^{t) + 1) - Ci(C'" - + 2) + go- C 2 . (5.5) 


The following asymptotic property of the sequence {T”} will play a key role. Recall T* defined in 
(4.5) and the fluid scaled processes T” introduced in Section 2.3. 

Lemma 5.8. As n ^ 00 , T" T* and Q” => 0. 


Proof. From the second expression in (4.10), we have for t > 0, 


Aruf,. _ Qm , Qm _ xtfit) x-{t) p 
^^^2 - —77 I -r- - I I- h Rh 


and thus 

Qm 




To To 

Qm xf{t) , x^{t) Qm 


To 


To 


P f lB(n.s)d7?(s) + f iB{n,s)‘^ dlf (s). (5-6) 

Jo Jo 


To To To To To 

Note that the last term on the right hand side of (5.6) is equal to 0 when t = 0, and is nondecreasing, 
and increases only when the term on the left hand side of (5.6) is 0. Therefore the left side of (5.6) can 
be represented in terms of the one-dimensional Skorohod map T (see Proposition A.2 (i)) and we have 
for t > 0, 


W^it) = T 


xm , ^ 3 ”(-) Q 2 (-) 


B(n,: 


+ / 1 


B{n,s)dl2{s) J {t) + ^ ^B(n,t)^- (5-7) 

To 


Ti To To 

Thus using the Lipschitz continuity property of the Skorohod map (see Proposition A.2 (ii)), we have 
for t > 0, 


sup 1 T 2 ”(s) < ^ sup P 

0<s<t 0<s<t 


xm , ^ 3 (-) Qm. 

- n -'- n - 

To To To 


lBiu,s)dl2is)]is) 


^ 1 C2(S). 

/rr ..n ^B(n,s)'= 

y/n o<s<t To 


< —= sup 

0<s<t 


< —= sup 

0<s<t 


xm) , xm) Qm). 

,,n ' ,,n ,,n B{n,s)^ 

Ms Ms Ms 


^B{n,u )^^2 (^) 


xm) , x^is) 


To 


To 


+ 


2 dlo logn 

Tm 


4” j — / ^B(n,u)di2 (^) 
ypn Jo 


1 dlo log n 

y/n gJfyfd 

dlo log n 

gjfn 


From functional central limit theorem for renewal processes, A" and S'" converge weakly to Brownian 
motions. Combining this with the fact that T'"(t) < t for n > 1 and t > 0, we have for all t > 0, 


— sup 
y/ri o<s<t 


xm , ^ 3 "(s) 


To 


To 


0, in probability. 


(5.8) 


Next from Corollary 5.6 we have for t > 0, / iB(n,u)dl 2 (u) —>■ 0 in probability. Finally, since 0 


as n —> oo, we have 


sup IF^(s) —7> 0, in probability. 

0<s<t 


(5.9) 
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Next using the representation for Vh" from (4.10), we have 

X^(f) X^(f) 

W^{t) - 'biW^it)) = ^ + I^(t) - nw,-(t)), 

which implies 

Ml M 2 

- - 'fmmicin.t)+iiit), (5.10) 

where C(n, t) = {Wi(t) — < go/\/n\- Note that the scheduling policy described in Definition 

3.2 is such that /" is equal 0 when f = 0, is non-decreasing, and increases only if the left hand side of 
the above equation is 0. Thus using the characterizing property of the one dimensional Skorohod map 
we have for t > 0, 

w^{t) = r (^ + ^ - (TTr(-) -'i'(w^2”(-)))ic(n..) I it) 

\ Ml M 2 / (5.11) 

+ (ITr(t) - + 4'(W2”(0)- 

By the Lipschitz property of T and 4' we now have. 


sup Wi{s) < —= sup 

o<s<t \/n o<B<t 


M? 


XUs) 

M^ 


+ ^ sup 'i'iW^is)) 

0<s<t 


3 

-I- - 1 = sup 

0<s<t 


(W^s) - ^mmicin,s) 


2 

< —J= sup 

y/n o<s<t 


X{s) 

M? 


^£(f) 

M^ 


-I-sup W 2 (s) H-• 

M2 o<s<i n 


As for (5.8), we have 

2 

—/= sup 
\/n o<s<t 

Using this along with (5.9), we now get 


X{s) , X2"(s) 


Ml 


M2 


—^ 0 in probability. 


sup lU"(s) —>■ 0 as n —>• 00 . 

0<s<t 


(5.12) 


(5.13) 


From (5.9),(5.13) and (4.10), we have Q" 0 as n —>■ 00 . Finally, using functional central limit theorem 
for renewal processes again, we have 


„-i/ 2 ^n(.) ^ 0 , n-i/25”(T”(-)) ^ 0, 


* = 1,2 j = 1,2,3. 


Hence, the conclusion follows from (4.1) and the fact that Q" => 0. 


□ 


The following theorem gives certain uniform integrability properties that will be needed to prove 
Theorem 5.2. 

Theorem 5.9. Fori = 1,2, 


and 


lim sup 

n—^oo 



2 


sup lU”(s) 

.0<s<t 


dt < 00 , 


(5.14) 


lim sup lim sup 

T—>-oo n^oo 





2 


sup wris) 

0<s<t 


dt = 0. 


(5.15) 
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Proof. We only prove (5.15). The proof of (5.14) is similar. For the case i = 2 the proof of (5.15) 
is identical to that of Theorem 4.11 of [4], with one modification. Unlike [4], here we consider arrival 
and service processes that are general renewal processes rather than Poisson processes. Thus one cannot 
directly apply Doob’s maximal inequality to bound ]E{ sup |A”(s)|} and E{ sup p"(s)|} .However, 

0<s<t 0<s<t 

these quantities can be bounded using Lorden’s inequality as in [1] (see Equation (172) therein) and we 
omit the details. Consider now i = 1. From (5.12) 


sup Wi{s) < 2 sup 

0<s<i 0<s<i 


^£(f) 


6/i3 


sup W^{s) + 


^J■2 0<s<t 


3go 


(5.16) 


The result for i = 1 now follows from the above estimate along with the property (5.15) for W^- □ 

We now prove the main result of this section, Theorem 5.2. 

Proof of Theorem 5.2. Let c,l be as in Corollary 5.6 and consider the sequence {T"} with threshold 
parameters lo ^ I, 9o > ^ c as above. From Lemma 5.8 and functional central limit theorem for 
renewal processes, it follows that 

X^^X, (5.17) 

where X is as introduced below (4.6). Define processes Zf, i = 1, 2 as 

X^(f) X^(f) 

znt) = - 'frnit)) - (lur(t) - 'smmncin.t), 




92 


. XJf{t) X^{t) Qm, , 

— —Jl -1- — -lB(n,s)«72 (S). 


93 


9i 


9i 


Also let T : D([0, oo) 


—7> D([0, oo) : R) be the identity map. From (5.7) and (5.11) we have for t > 0, 
jr(t) = (F-i)(ir)(t), (5.18) 


Pf{f) = {T-T){zq){t) + / lB(n,s)dI^{s). 


(5.19) 


Next, from (4.6), 


X 


_2_ 

Ms 


vn V V vn vn 

,,n ,, ' ,, ,,n ' ,,n 

Ms Ms Ms Ml M 2 


91 92 


(5.20) 


Applying (5.20) and the third convergence statement in Corollary 5.6 to (5.19), and recalling 
defined in (4.24) and (4.26), we see that 


Also from Lemma 5.7, for some ci,C 2 G (0,oo), 

\wf (t)-^ {Whitman,t) 




< 


90 


y/n 
9i 


1 - 


939i 


9392 


<3r«)+^ 

92 


1 - 


9392 


From Assumption 2.2 we have 


^/n 


1 - 


9392 

9392 


{ 2 Qlf{t) + -) 
n 


1 ci(C'"-L" + 2) 


\/n 


9392 


-)> 


|g3^3 — ^ 2 ^ 2 ! 
93 


and from Lemma 5.8, Q" 0. Using these observations in (5.22) we see that 


+ ^. (5.22) 

vn 


(5.21) 


(5.23) 
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Applying the above result and (5.20), (5.21) to (5.18), we now get (W'”,/f) => {Wi,Ii). In fact we 
have shown {W'^, /") => (W*, I*) in I1([0, oo) : R"^). This proves part (i) of the theorem. For the second 
part of the theorem, we observe that from Theorem 5.9 and first part of this theorem, 


e-^‘E(lF”(t))dt ^ / e-'<*E{W*it))dt, i = 1,2. 


(5.24) 


From (4.10) we see that 


A{n,nt) = <{ Q^{t) - <^} = i ^ 




yjn 


\fn 


Combining (5.24) with Lemma 5.3 and using the definition of Q% we now have exactly as in the proof 
of Theorem 4.2 of [4], 

^OO ^OO ^oo 

/ e-^‘E(Ai^lFr(t)lA(n.„t)e)di+ / e-'^*E(M^tF2"(t)U(„,„,))dt ^ / e-^‘E(Q;(t))dt. (5.25) 

JO JO JO 

Finally using (5.24), (5.25), (4.10) and Corollary 5.6 it immediately follows that 

nOC) poo 

/ e-^‘E(Q”(t))d<^ / e-'^*E{Q*{t))dt, i= 1,2,3. (5.26) 

Jo Jo 


The result now follows from the definitions of J^{T^) and J*(0). 


□ 


6. Proofs of Theorems 5.4 and 5.5, and Lemma 5.7 


We begin with the following large deviations estimate for renewal processes, which will be extensively 
used in our proofs. In the form stated below, the result can be found in [2] (see also [1]). 

Lemma 6.1 (Lemma 6.7 of [2]). Let ® sequence of independent strictly positive random 

variables, where identically distributed with finite mean Ijv, and rji may have a different 

distribution from rii,i > 2. Assume that there is a nonempty open neighborhood O o/O G R sueh that 

A{1) = logE(e*''’) < oo for all I € O and i >2. (6.1) 


For eaeh n € N, let > 0 be such that lim„^.oo = v, and for each n £ N and i = 1,2,3,..., let 
rff = -^r]i. Given e £ (0,n/2), let n({v^},€) £ N &e such that when n > n({i'^},e), 


, n , 1 ^ 1 1 1 1 

\v^-v\<e, -^ . < 






1 + 


2 (^" — e) J V 


> 




> 


1 


For each n £ N and t > 0, let 


iV”(t) = sup < fc > 0 : y] 7?," < t 


Let 


i=l 


A*(x) = sup(/a; — A(/)), x £ 
zgr 


be the Legendre-Fenchel transform of A. Let Qi{n,e) = A* i +-^- ) © 2 ( 1 ^, e) = A* (i (l + jp)). 

Then Qi{v, e) > 0 for i = 1,2, and for n > n{{v'^}, e) and t > 2/e, 


P(iV"’(t) > (i/"- + e)t) < exp (-[(j^" + e)t - l]0i(i7, e)) < exp {-{lyt - l)0i(7^, e)), (6.2) 
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and for n > n({i/”}, e) and t > 0, 


P(7V"(t) < (i/" - e)t) < exp (-(i." - e)t02(z., e)) +F 

< exp {-{v - 2 e)tQ 2 {i', e)) + P , 


(6.3) 


Furthermore, if rji has the same distribution as r]i,i > 2, then for each n > l,t > 0, and 0 < po G O 
and any to S N, 

P ^.jnax^ry’" > < mexp exp(A(po))- (6.4) 

For k = 1,2 and j = 1, 2, 3, denote by A* ^ and A*^^- the Legendre-Fenchel transform of Aa^k and Asj, 
respectively, where the latter functions are as introduced in Assumption 2.3. These transforms along 
with Lemma 6.1 applied with sequences {vj{i)/pj}i^n, {uk(i)/Xk}iGn, j = 1,2,3, fc = 1,2, will play a 
key role in the proofs of Theorems 5.4 and 5.5. 

Let {Gi{t)}t>o be the filtration in Section 2.2 associated with the sequence {T”}. We next introduce 
a family of {C/"(t)}-stopping times as follows: For fc S N, 


T-o” = 0, 

r^Vi = inf : Q-{t) - ^Qf{t) > , 

= inf |t > : Qlfit) - ^QUt) < < C" - l| . 


6.1. Proof of Theorem 5.4 

Throughout this section {T"} will denote the sequence of control policies in Definition 3.2 with some 
choice of threshold parameters. Note that on ^ G Ni QsO) starts from below C” — 1 , and 

whenever Q 3 (•) becomes larger than or equal to C” — 1, Server 1 stops serving Buffer 2, which causes 
Qg (•) to decrease. Thus we have 

Qsit) < C", for all t G [T 2 k- 2 ’^ 2 k-i)’ ^-nd fc G N. 

Since k > C" < k{C^ — L" + 1) and so we have for s G [T 2 k- 2 ^''' 2 k-i)^ fc G N, 

g^(s)lA(„.s)<K(C"-L" + l). (6.6) 

We also note that for s G ['T 2 k- 2 ’'’' 2 k-i)^^ ^ Qsi^) ~ ^Qi(^) < Thus 

Ql(s)lA(n,s/ = 0. (6.7) 

In view of ( 6 . 6 ) and (6.7), to estimate P(£(n,t)), it suffices to focus on [T;^fc_i,T^fe), fc G N. Note that 
each corresponds to a up-crossing of Qg (•) — ^Qi(-) from smaller than L” to become larger than 

or equal to L”. Each up-crossing requires at least one service completion of Server 1. Let 

k” = [nt(A”-bA” + 2)J-hi. ( 6 . 8 ) 

Then we have that 

P(r 2 V-i < nt) < F(S"(Tr(nt)) + S"(T"(nt)) > k") 

< F(A^(nt) + A"(nt) > k”) 

< F(A^(nt) + A^(nt) > nt{Xf + + 2)) 

< P(A"(nt) > nt{Xf + 1)) + P(A 2 (nt) > nt{X 2 + 1)). 
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From (6.2) in Lemma 6.1, when n > max{n({A"}, l),n({A 2 }, 1)} and nt > 2, we have 


IP’(''' 2 k"-i ^ ^ 6xp |-(Aini - l)0“’^(Ai, 1)| + exp |-(A 2 ni - 1)0“’^(A2,1)|, (6.9) 

where for e > 0, 0“’*(Ai, e), j = 1, 2, are defined as 0i in Lemma 6.1 on replacing A* with A* j. Also, 
P(r2V-i > nt, £{n,t)) 

<^P( sup Q^(s)U(n..) >K(C"-L" + l),r2Vi<^i) 

k=i / 

+ sup Q)‘(s)lA(n,.)c >«(C"-L" + l),r2Vi <nA 


< P (Qa (s) > k(C'" — L" + 1) and A{n, s), for some s S ['’' 2 k-n'^ 2 k ^ nt] ) 

k=l 

k"- 

+ ^ P (Qi (s) > k(C" — L" + 1) and A{n, sY, for some s S '^ 2 fe ^ nt] ) 


( 6 . 10 ) 


k=l 


Next, for each A: £ N, we define a sequence of stopping times within [T^k-nT^Y)- Let 


For I £ N, 


H{n, t) = ^ QUt) > ||(C" - L" + 2) }>, G(n, t) = {Q^ > (C” - 1)}. 


='r 2 fc A inf |t > 77”’^2 : (A(n, t) n G(n, t)) or (A(n,t)'= n ff(n,t))|, 
= '^ 2 k A inf > 7?2i-i : A{n,tY n ff(n,t)‘=| . 


( 6 . 11 ) 


Note that 


{uj :t € [’72/-2> ^ 2 /-i)} {^in,t) fl G{n,t)Y H {A{n,tY H H{n,t)Y fl {A{n,t) fl G(n,t)'^}‘^ 


= A{n,tYnH{n,tY. 


( 6 . 12 ) 


Also, there exists ni £ N such that when n > m, we have Since k > we have from (6.12) 

that when n > ni, 

I sup Q^(s)U(„,,) >«(G"-L'^ + 1)|=0, 

^ ^ r n.k n ,k \ J 

I sup Q)‘(s)U(„,,)c>«(G"-L’^ + l)|=0. 

^_rn,fcn,fc\ J 


Thus to estimate V{£{n,t)), it suffices to consider the intervals G N- We now estimate 

how many such subintervals are within [T^k-i^T^k A nt). We observe that each corresponds to at 

least one additional arrival to Buffer 1 or one additional job completion for Buffer 3. Recall k” defined 
in (6.8). As in the proof of (6.9) we have, for all n > max{n({A”}, l),n({A 2 }, 1)} and nt > 2, 

< nt) < P(A"(nt) + S^iTYint)) > k") 

< F{A^{nt) + A”{nt) > k") 

< exp |-(Aint - l)0“’^(Ai, 1)| + exp |-(A 2 nt - 1)0“’^(A2,1)|. (6.13) 
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Thus from (6.9), (6.10) and (6.13) we have for n > max{ri({Ai}, l),n({A 2 }, l),^i} and nt > 2, 




F{S{n, t)) < ^ ^ P (a) > /t(C^ — + 1) and A{n, s) for some s G [V 2 i-i^V 2 i^ ^ ) 

k^l1^1 
k”^ k”^ 

+ ^ (Qi (s) > k(C'" — L" + 1) and A{n, s)° for some s G Vhi-i^''hi^ ^ ) 

k=l 1=1 

+ (k" + 1) ^exp |-(Ainf - l)0“’^(Ai, 1)| + exp |-(A 2 nf - 1)0“’^(A2: 


Next, on the set {Qg (s) > k(C'" — L" + 1) and A{n, s)} 


g"(s) > - L”) > - L" + 1) - L") 

M2 


> 


> 


fii 3 k 
3 k 


((^n _ 2," ^ + 1) - i”) 

/r2 V 4 / 

(C" _ L" + 1) + || (J (c - 1) - l) Zo logn 
(C^-L^ + 1), 


where the last inequality follows on noting that k > Let n 2 € N be such that when n > n 2 , we 

K, Ml ~ 

2 p,2 ' 

Q^{s) > k'(C" - L" + 1). 


have Letting k' = min{K, f ^}, we have that, when n>n 2 , 


Note that since k > max{^^,4}, we have that 


K > -. 

M2 


(6.14) 


Since k > k', for n > max{n({A”}, 1), ndA^}, 1), ni, 712 } and nt > 2, 

k'^ / 

P(£(n,t)) < 2 EE P f Qi{s) > k'{C^ — L" + 1) for some s G A nt) 

k=i 1=1 ^ 

+ (k” + 1) ^exp |-(Aint - l)0“’^(Ai, 1)| + exp |-(A 2 nt - 1)0“’^(A2,1 


(6.15) 


Next note that 


{A{n, s)^ n H{n, s)'^}'^ fl {A{n, s) fl G(n, s)'^}'^ 

= {A{n, s) n G{n, s)} U {H{n, s) fl G(n, s)} U {H{n, s) fl A(n, s)^} . 


(6.16) 


From this, we see that for s G [?? 2 /h’^ 2 /^)’ Ci(®) — — L" — 1) > 0, and Server 1 works on 

Buffer 1 continuously. Also, from (6.12), we have Q"(T? 2 i’A~) < — L" + 2), and so 0"(?72/’A) ^ 

[^(G" — L" + 2)J + 1. Using this fact, the probability in (6.15) can be estimated by analyzing a 
G//G//1 queue with interarrival times m", u"(A:), fc = 2, 3,..., service times w", u”(fc), fc = 2, 3,..., and 
initial queue length [^(G" — L" + 2)J + 1, where ft" and u” are residual interarrival and service times 
at time Buffer 1. Let {Q^{t)}t>o be this GlfGIfl queue length process and dehne 

;9" = inf |s > 0 : Q"(s) < ^(G” - L” - 1) 

I M2 
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Noting that when n > ni, ^ and using (6.14), we see that 

P (Qi (s) > k'{C^ — L" + 1) for some s S ^ 

< P (Q"(s) > k'(C" - L" + 1 ) for some s £ [0, ^"] ). 
Let e G (0, (/ri — Ai)/4), and define 

- L" - 1) - 2 J 


(6.17) 


f" = 


2(A? + e) 


Let and S’" be the arrival and service processes of the GljGIjX queue, and consider 

U" = + e)r, 5"(r) > (/x” - e)r} . 

From (6.2) and (6.3) in Lemma 6.1, for n > max{n({A"}, e), n({/x"}, e)}, and G > 2/e, we have 
P((H”)=) < P(i’^(t”) > (A^ + e)t”) + P(5'"(r) < (ax? - e)r) 

< exp (-(Alt” - 1)0 ?A(Ai, e)) + exp /(m - 2e)r02’^(/xi, e)) + P ^0? > 

where 02’^ is defined as 02 in Lemma 6.1 by replacing A* with A* 

Next using (6.2) and (6.4) in Lemma 6.1, 




cG 


< 


cG 

max z;?(fc) > 
fc—l,...,S’^(ni) 


< P {Si{nt) > (ax? + e)nt) + ] 


fG 

max Vi{k) > ——- 

fc—l,...,[(Xi5^+e)nXj 


(6.18) 


< exp/{flint - l)0?^(/xi,e)^ + (ax? + e)ntexp exp(As,i(po)), 


where 0?^ is defined as 0i in Lemma 6.1 by replacing A* with A* i and 0 < po € O. Thus for 
n > max{n({A?}, e), n({Ax?}, e)}, and G > 2/e, we have 


P(('H")“) < exp /{\iG - l)0?’^(Ai,e)) + exp /{pi - 2 e)t” 02 ’^(AXi, e)^ 
+ exp / {flint - 1)0?^(axi, e)^ + (ax? + e)nt exp ' 


2axi 


exp(As,i(po))- 


(6.19) 


Next note that on H”, 

Q"(r) = Q"(0) + i”(t") - S^{e) 

M? 


< ^(C” - L” + 2) + 1 + (A? - AX? + 2 e)t” 
1*2 


= ^(C” - L” + 2) + 1 - (ax? - A? - 2e) 
1*2 


/{G^ - L" - 1) - 2 J 


2(A? + e) 

Recall that e € (0, (^i — Ai)/4). Thus there exists ns gN such that when n > 723 , we have 

n 

Ml 


gn(^n)_ A^(^n_^n_i) 

M 2 

Q/.n 

<-^ + l-(AX?-A?-2e) 
1 *? 

< 0 . 


^(C" - L" - 1) - 2 


2(A? + e) 
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Since Q^{t) > — L" — 1) — 1 for all t € [0,/3"], we see that for all n > n^, on 'H", f" > /3". Next, 

on this set, for s G [0,/3"] and n > 713 , we have 


Q"(s) = Q"(0) + i"(s)-^"(s) 

< Q"(0) + i"(s) 

< ^(C” - L” + 2) + 1 + (A^ + e)r 

^2 

I in ij'^ 

< _ 7 ^" + 2) + - L" 


M 2 

M 

2m^ 


(C" - L” + 1). 


1 ) 


Recall from ( 6 . 14 ) that k' > and so there exists 714 > 713 such that when ti > 714, we have /c' > ||hr, 
and so for such n, on "H", 


Q"(s) < k'{C'^ - L” + 1) for all s G [0,/3”]. 

Let I" = {Q"(s) > k'(C'"-L" + 1) for some s G [0,/3”] }. Then for ti > 774, P(-H”nl") = 0, and so for 
n > max{7i({A"}, e), 7 i({A 2}, e). Til, 774}, the probability in (6.17) is bounded by P(('H")'^). Combining 
this with (6.15) and (6.19), we finally have for sufficiently large 77 and t > 2 / 77 , 

P(£’(77, t)) < (k" + 1) ^exp |-(Ai77t - l)0“’^(Ai, 1)| + exp |-(A277t - 1)0“’^(A2,1)|^ 

+ 2(k")2 exp (-(Alt" - l)0“’'(Ai, e)) + 2(k")2 exp (-(/n - 2e)r0^’'(77i, e)) 

+ 2(k"')^exp(^-(77i77t- l)0i’^(77i,e)^ +2(k")^(77i +e)77texp exp(As,i(po))- 

( 6 . 20 ) 

Thus we have shown that, there exist Oi, 82 , 63,64 G (0, 00 ) such that for sufficiently large 77 and t>2jn, 
we have 

P(iSi( 77 , t)) < 0 i{nt + 1 )^ ejip{— 02 {nt + 1 )} + @ 3 ( 77 ^ + 1 )^ 77 “®'*^°“^^*°. 

□ 


6.2. Proof of Theorem 5.5 


Throughout this section {T"} will denote the sequence of control policies in Definition 3.2 with some 
choice of threshold parameters. 

We now introduce a sequence of {Qi {t)}t>o stopping times, which will be used in the proof of Theorem 
5 . 5 . First recall the stopping times rf,! G Nq in ( 6 . 5 ). As in [ 4 ], for A: G N, we define a sequence of 
stopping times within [T 2 k- 2 ^^ 2 k-i)- ^or d,lo > 0, let D” = d/olog 77 . For 777 G N, 


M2m-i = min|r2Vi,inf{s > V2m-2\QUs) > C" - 1}| 
= min|r2Vi,inf{s > (s) < C" - 1}| 

/3m'' = min|i72;,^_i,inf{s > ?72m-2lQ2(s) > ^}| ■ 


Recall the multi-parameter filtration {t/"(a, 6 ) : a G N ^,6 G N^} and multi-parameter stopping times 
introduced in Section 2.2. Lemma 6.2 below is taken from [1]. 
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Lemma 6.2 (Lemma 7.6 of [1]). Let T = (Ti, T2, T3, T4, T5)' be a {C/"(a, 6) : a G G N^} 

multi-parameter stopping time. Then 

and on {T G N®} the conditional distribution of {{ui{Ti-\-k), ulf {T2+k), vf (T3-\-k), vf {T4+k), W3 (T5 + 
k)y : fc G N} given is the same as the distribution of {{ui (k), U2 (k), Vi lk), V2 {k), v^{k)y : k G N}. 


The following lemma follows along the lines of Lemma 7.5 of [1]. The proof is omitted. 
Lemma 6 . 3 . For n,k,m G N, 




is a b) : a G b G N^} multiparameter stopping time. 


Proof of Theorem 5.5. From the definition of stopping times in (6.5) we see that, for S e [T2k-l^'^2k)^k G 

N, 

Qsis) - 4Qi(s) > i”, or Q^is) > C" - 1, 

Ml 

and therefore for such s, Q 3 (s) > 0. Consequently, since Server 2 does not idle unless Buffer 3 is empty, 
we have 

/ l{Qj(s)>n"}cZ/^(s) = 0. 

Thus we only need to consider time intervals [T 2 k- 2 ’''' 2 k-i)’^ ^ We observe that for s G 
Qsis) > C" — 1 > 0. Thus Qsis) = 0 is possible only for s G lli 2 m- 2 ’We next estimate how 
many such subintervals are within [t^j ,_2 A nt,T 2 /._i A nt). Each 772^-1 corresponds to at least one 
additional job completion for Buffer 3. Let k" = YfpLf + l)n<J + 1. From (6.2) in Lemma 6.1, we have 
when n > n{{p. 2 }, 1) and nt > 2, 


^(^ 2 /-! ^ < ^{S^int) > k”) < exp /{p 2 nt - l) 0 i’^(/r 2 ,1)^ ■ (6.22) 


Let 

TZ{ri,t)=\l l{Qj(s)>_Dn}d/^(s) 0 I . 

[d[0.nt) J 

Then from (6.22), we have when n > max{n({/i 2 }, 1)}, and nt > 2, 

P(7e(n,t),r2V-i > nt) < f f l{Qj(s)>U"}cZ/r(s) ^ 0^ 

fc=i \“'ff2’ic-2. / 


k" k? / ^ 

k k ^{Q^{s)>D«}dl2{s) 

k=lm=l \“'[') 2 m- 2 .^ 2 m-l'^"*) / 


(6.23) 


+ k” exp /{fi 2 nt - 1 ) 0 ^’^( 712 , 1 )) ■ 


By the definition of for s G [fj 2 m- 2 ^ ^ QEi^) < so a typical summand in (6.23) is 

equal to 


+ P = 12 ^- 2 , 


f/ 2 y^_i Ant) 


l{Qj(s)>n"}d-f?(s) y 0 




l{QJ(s)>r>"}C^/2 (s) 7^ 0 . 


(6.24) 

(6.25) 
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In the following, we estimate the probabilities in (6.24) and (6.25) separately. We will use the following 
constants: 

^ fn ■ ^2 - MS M2 Ms 

ei e 0, mm ■!. 1,---, —, - 


(6.26) 


2 32cms 

There exists ni G N such that when n > ni, also noting that A 2 = Ms from Assumption 2.2, we have 


Ms 


M 2 - Ms 


1 M 2 - Ms 
= mm < — 


M2 + Cl < 2/i2, M3 + Cl < 2/i3, A2 + Cl < 2A2, M2 ~ Ms ~ 2 ei > 


M 2 - Ms 


M2 ”•^2 4" 2ei < 2(^2 ~ A2). 

For the rest of the proof we assume n > ni. Define for s > 0, 

i^(s) = A-(C’'= + s) - A-(C’'=), sns) = +s)- 5r(7^”(/3”’'=)), * = 2,3. 

Now denote by the event in (6.24), i.e., 


t 2 n,k ■ J on.k 

B’ = < > 


— /1, h. 

%m-2i 


42m_lAnt) 




We claim that, with = {supo<,<e^ |< 32 (/ 3 m + s) - ^| > 5^}, 

P n < exp |-(^^ - 1)0 ?’^(m 2, ei)| + exp {-(^^ - 1)0?’^(A2, d) 

To see this, note that 


1 

P n B-J) < P ( Q^(/3"’'= + s) < -— for some s S [0, ^], B^J 

3 

+ P ( + s) > -— for some s e [0, ^], B'^^ 


By the definition of we have Q 2 (/^m'") > ^- For + s) < for some s G [0, ^], we 

need at least service completions for jobs in Buffer 2 in units of time. On the other hand, 
noting that > M 2 m- 2 . we have ^ and so Qa (/^m'') < -^ + 1. In order for + s) 

to be greater than or equal to for some s G [0, we need at least -1 arrivals to Buffer 2 

in time. Therefore, we have 

/ f)T)^ 1 T)^ \ / f)D^ 1 

< P (^2 ( —) > (^2(-^) > 2X ■ 


Next note that 




(6.27) 
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From (6.2) in Lemma 6.1, Lemmas 6.3, and 6.2, and a conditioning argument, when nt > 2/ei and 
n > inax{n({/i^}, ei), n({A^}, ei), ni} 


,90^- 


1 


K 


2 4 


< nt +P > - — ,[3^'^ < nt 


,9D^ 


ID’^- 


K 


2 4 


OTl^ f) 

< P(^2(-^)>(/^2+ei)-^ 


f)Tl^ fiT)^ 

+ P(A^( —)>(A^ + ei) — 


< exp<^ -( 




K 


- l)0i’^(Ai2,ei) ^ +exp<^ -( 


,A 26 »iA" 

K 


-l)0“’"(A2,ei) . 


This proves the claim. Next, 




f) V)'^ r)^ 

^n,k on.k C'-T/ J-y /^n,k\c yyn.l 

V2L-I - Prri - 

( /9 

T)^ ' 

sup + s) < - —, 

tv A 0<s<^ 2 4 ^ 

/ n r^n r\n 1 y^n \ 

+ p - -^ > ir>+ s) > -—, 6^’'= 


(6.28) 


(6.29) 

(6.30) 


On the event in (6.29), + s) < for all s € [0, and + s) > Z?” for some 

s S [^^,V 2 m-i ~ /3m^] C Thus there must be at least arrivals to Buffer 2 in 

time, and so (6.29) is bounded by 




ID" 

> - 

“24 


From (6.27), and using similar argument as in (6.28), we have the following upper bound for the 
probability in (6.29), 




K 


D" 


^2 DF >(A^ + ei)^ <exp -( 


K 


,A2D" 

K 


-l)0“’'(A2,ei) 


Next consider the event 

A-l'^ = U 


1=2,3 


5" 


(0 + 1)D^ 

K 




(0 + 1)D" 


K 


> ei 


(0 + 1)D" 
K 


Then 


p(Ai;^")< E 

z^2,3 

i=2,3 


5" 


(0 + l)D^ 
K 

(0 + l)D 
K 


^ (m? + Cl) 


(0 + 1)D" 


< (m” - ei) 


K 

(0 + l)D 


K 


(6.31) 


(6.32) 


From Lemmas 6.2, 6.3 and (6.2) in Lemma 6.1, when n > max{n({/r 2 }, ei), n({/r 3 }, ei)} and nt > 2/ei, 


E’ 

i=2,3 


(0+ 1)D" 
K 


> {Pi + ei) 


(0 + l)D" 
K 


< j: exp A('df±l)5_ _ i)er(ft,..) y 


i=2,3 


K 
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Let V 2 and v'^ denote the residual service times for the jobs in service at time in Buffers 2 and 3, 
respectively. Then from (6.3) in Lemma 6.1, and once again using Lemmas 6.2, 6.3, we have 


E' 

i=2,3 


( 6 »+ 

K 


< exp ( -(^ii - 2ei 
2^2,3 


< - ei) 

ie + i)D 


{9 + l)D^ 
K 


K 


ei) j + ^ 1 

i=2,3 


> 


ei(6l+ !)£»" 
2tifK 


Now using (6.4), similar to (6.18), we have for i = 2,3, 

P (/i > ^ ^ exp /{flint - l)el’\fii, ei)) 

+ (m" + ei)ntexp ) ^^"pAsApo)), (6-33) 


where 0 < po G O. Combining the above estimates, we have 


PMS‘) < y: exp] - i)er(ft,e,) 


2=2,3 


K 


+ exp E(/r* - ^ exp - l)0i’*(//i, ei)) 

2=2,3 ^ 2=2,3 


E {p 7 + ^i)ntexp [ - 


2=2,3 


Poei{0 + 1)D^ 


2piK 


exp{As^i{po)). 


(6.34) 


We next observe that on the intersection of and the event in (6.30), for s G [0, 

+ s) is always nonzero, because Q^iPm^) > 0^/4 and 


(6» + l)Zl’^ 
K 


, „ N6»+ !)£)" „ 2D" D" 

< (1^2 + ei)-^^ ~8 


Also on this set, ^ 2 m -2 ^ 


K 


+ P'A" < V 2 m-i and so 

< c--1. 


(6.35) 


Consequently, according to our policy. Server 1 works on Buffer 2 continuously over the interval 
]. Thus, on this set. 




>5” 


(6» + 1)D" 
K 


- Stt 


(0 + 1)D" 


K 


^ (a*2 Ms 2ei) 


(6» + 1)D” 
K 


> 


M2 - M3 -P" 
2 K 


> C". 


However this contradicts (6.35) and so we must have that the intersection of {A^Y and the event 
in (6.30) is empty. Thus the probability in (6.30) can be bounded by P(v4((]^), and combining this 
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observation with (6.34) and the bound on the probability in (6.29) obtained in (6.31), 




,X2D 


- 1 -l) 0 ^^(A 2 ,ei) \+ ^ exp<{ -( 


+ exp ( -(fXi - 2 ei) 
2^2,3 


2=2,3 




K 


-l)0r(M*,ei) 


(6» + !)£>’’ 


(^i” + el)^^^exp f- 
2=2,3 ^ 


K 

Poeii9 + l)n 

2fiiK 


i=2,3 

exp(As,i(po))- 


We now consider the event in (6.25), and denote it by C”’'', i.e.. 


28 


exp - l) 0 ®’*(Moei)) 

'' 6 — 0 O. 


(6.36) 


^n,fc } on,H _ -n 

\ Pm w2 


n.k 




^{Q^(s)>D^}dl2{s) 0 > . 


When fc = m = 1, on C"’^^ we have ,5"’^ = = 0, and so = 0, which is a contradiction to 

the definition of /3"’^. Thus P(C”’^) = 0. Consider {k,m) ^ (1,1). Let 


q_^n,k _ 


rn -| jjn 

inL. + s) < ^ or inf + s)<-—\. 


0<s<^ 


We now estimate fl Observe that on {?72m-2 < 3-;^fc-i A nt}, for /c > 2, m = 1, 

Q3ifl2i-2-) = Q3{r2k-2-) > min{L", C" - 1} = L\ 


and for fc > 1 , m > 2 , 


QUV2i-2-) > C" - 1 > L". 


Thus for {k,m) / (1,1), on we have — Q 3 (^ 2 m- 2 ) > — 1- By the definition of , 

we have For + s) < 5 -^ for some s G [0, we need at least service 

completions for Buffer 2 in time. Similarly, to make Q 3 (/ 3 (),A _(- g) < A- for some s G [0, we 


need at least service completions for Buffer 3 in time. Thus we have 


K 


,90 


P(2^()^'=nC”’'=)<P 53"(—)> — +P 4"(—)> 


K 


,9D^‘ 


K 


ID” 

2~ 


Noting that 


F” „ 26»d^„ , „ ,29(0”-d) , „ ,9D” ID” 2^290” , ^ ,90 

— > 2/i3^F" > K 2— - 


K ’ 


we have when n > maxjnd/ig }, ei), n({/i 2 }, ei), ?t-i} and nt > 2 /ei 

.,/c nn.k 


f)r)ri 

< p ( ssi—) > (p-, + Cl)— 


n T~)n n r^n 

+ F[S^{^)>{p” + e,) — 


< y]exp<{-( ^*^ -i)0i’d/xz,£i; 


2=2,3 


Next, 


Fi{n^>^r n C-'=) < p - 


^/wn Ton 

■ ,k on.k u /njn,k\c nn.k 

^ ’ V^m / ' ' 

n Ton Ton 


(6.37) 

(6.38) 

(6.39) 
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Using a similar argument as for (6.30), we see that 


29 


^2m—1 f^m 


0 T)^ 

" > IT’ ^ 


(6.40) 


We now consider the probability in (6.39). Define to = and 


N = 


D" 

K 

to 


1 G 


8cfJ,3 
M2 - M3, 


+ 1) 


16 c/i 3 
M2 - M3, 


For s G [0, to], j = 2,3 and / = 0,..., tV — 1, define 


fir)n \ \ / / GT)^ 

5",+i(s) = 5 ; ( t; (+ — + tto j + s j - (t;" f + it + 




^2.l+l{^) — ^2 ( + /to + S j — ^2 ( /3m^ + + ^^0 




Consider the event 


pn,fe _ 


*^3,1 (^o) Ma/'O 


> ei or 


S2,l{to) - M2^0 


> £1 or 


^2,i(/o)-A^to 


/o 


> £1 


We first estimate P(£l^u)- Clearly, 


p(Cp) < E 


J=2.3 


5"i(to)-M”/o 


/o 


> £1 


A"i(to)-A^to 


to 


> £1 


— E] (*^yi(/'o) > iC + ei)/^o) + P (^2,i(/o) > (^2 + ei)to) 


i=2.3 


+ E ^{Sjito) < (m” - £i)to) +P(Gl”(to) < (A^ - £i)to). 
i=2,3 


Now from Lemmas 6.3, 6.2 and (6.2) in Lemma 6.1, we have for 

n > max{n({/x^}, £i), n({/i^}, £i), n{{Xl^}, £i), ni} and nt > 2/ei, 

E/ ^ ('^yi(^o) > (Mj + £1)^0) + P (^ 2 ,i(/'o) > (A2 + ei)/o) 

1=2.3 

< E exp |-(/rjto - 1 ) 01 ’-^(/rj,£i)| + exp |-(A 2 to - 1 ) 0 i’^(A 2 ,£i)| . 

1=2.3 

Let DJ) ^3 denote the residual service times for jobs in service at time _|_ Buffers 2 and 3, re¬ 

spectively, and U 2 the residual arrival time at the same instant for jobs to Buffer 2. From (6.3) in Lemma 
6.1, and Lemmas 6.3, 6.2 again, we have when n > max{n({/i 3 }, £ 1 ), ndfi^}, £ 1 ), ?t-({A 2 }, £ 1 ), fi-i}) 

E P (5”i(to) < (m” - ei)to) + P (A” i(to) < (A^ - £i)to) 

1 = 2.3 

< E exp (-(mi - 2£i)to02’^(i/j,£i)) + E ^ 

1 = 2,3 1 = 2.3 \ ^^1 / 

+ exp (-(A 2 - 2 £i)to 02 ’^(A 2 , £ 1 )) + P (^ua > ■ 
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Using similar arguments as in (6.18) and (6.33), we have for j = 2, 3, 

P l)0i’'^(A^j.ei)) + ifi]' + ei)ntexp 

( > < exp /{Mnt- l)0“’^(A2,ei)) + (A2 + ei)ntexp 


30 


and 


Poeifo 

2fj,j 

PoeUo 

2 A 2 


exp(As,j(po)), 


exp(Aa, 2 (po)), 


where 0 < po & O. Thus 

P(Cp) < exp|-(/ijdo - l)0i’^(pj,ei)| +exp|-(A2to - l)0i’^(A2,ei)| 


i=2.3 


+ Yl exp (-(Pj - 2ei)to02’^(Pi,ei)) + Y exp (-(p^nt - l)0i’^(pj, ei)) 

j=2,3 j=2,3 

+ Y (Pj + ei)”-^exp exp(Asj(po)) + exp (^-(A2 - 2ei)to02’^(A2,ei)) 

j=2,3 Pi 1 

+ exp /{X2nt - l)0“’^(A2,ei)^ + (A2 + ei)ntexp 


Poeitp 

2 A 2 


exp(Aa, 2 (po)) 


= r(to,ni). 


On the intersection (f”(^)'^ and the event in (6.39), we have > 

and for s G [0,to], 


QS 



0D^ 

~ir 



>Ql 


3n,k 


0D^ 


1 H" 

> 5 --(.5 


> 


ID 
2 4 2p3 

1 P2 \ 


K 

- ei)fo 

+ ei L” 




8 2pL^d J 


2 

D" > 0, 


^2”l(s) 


(6.41) 


where the last inequality follows from noting that K > Since Q 2 is nonempty on the time 

interval _(_ iR —h tp], Server 1 will work on Buffer 2 continuously during this time 

interval. Consequently, we have 


\ / f)n^\ 

+ — +toj>Q^ + — j + Sl.ito) - Sl,{to) 

r n 

> — + {p 2 — 0-3 ~ 2 ei)fo 
L" 

> — . 

- 2 

Furthermore, for all s G [0,tp], 

( f) T)^ \ / f) T)^ \ 

+ ^ + /=Qs + — j + -s^.is) 

> -(p3 + £ 1)^0 

_ L" fj.^ + ei L’" 

2 2^3 2 

> 0 . 


(6.42) 


(6.43) 
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We repeat the above analysis for the time interval [to,2to]. First define the event as by 
replacing 5'"i(s) and ^ 2 .i('S) with ^ 2 , 2 ( 5 )- We then obtain that for 

n > max{n ({^3 }, ei), n({/r 2 }, ei), ei), ni} and nt > 2/ei, 

^(^rn’, 2 ) the same upper bound r(to,ut) as P(£’^’^^). On the intersection O (£m/‘^ and the 

event in (6.39), following the similar arguments to those in (6.42) and (6.43), it can be shown that 




L" 


K 

and similar to (6.41), we have for s G [0,to]) 


9D^‘ 


Q 3 ( 4- 77 -f 2to ) > Q 3 I Pm^ 4- 77 -h to 4- S I >0, for S G [0, to]) 


K 


OD^- 


Q2 ( + fo + S )> Q2 ( C’" + — + fo ) + ^2.2(5) - Sl2{s) 


,8 /rad 


K 

> 0 . 


Repeating this argument N times we see that on the intersection and the event in (6.39), 

for s G [0, to], and t = 0,..., — 1, 


OD 


Qs ( Pm^ 4- — -h tto + S ) > 0 


K 


(6.44) 


and 


Q2 ( p:^^ + ^ + ito + s) >q^(^p^^ + ^+ 

> 0 , 


> 


90 
K 

1 {I + 


^8 2113d 

where the last inequality follows from noting that K > 32/^2 4- dt£sl^zi£2l_ 

We note that, since Server 2 does not idle if there are jobs in Buffer 3, (6.44) implies that the 

-.Af (cn,k\ 

N 

(yn,kYr^C^,k] < 

K - K 


probability of the intersection and the event in (6.39) is 0. Thus 


f) 7 ~)^ T^n 

^ n< Ep(C') < mto, m). 


Z=1 


Combining the above estimate with (6.37), (6.40) and (6.34), 

P(C’'') < E *^xp|-( ^'^ - l)0i’*(/Xi,ei) I + E expE( 


2=2,3 


2=2,3 


K 


-l)0r(M.,ei) 


E “ 2ei)Ei:ElL0^-*(^.^ e^)) + ^ exp - l)03’*(/r2, d)) 

/ 1 — 9 


2=2,3 


4 - E + (^i)ntexp ( - 

2=2,3 


Poeii9 + l)D^ 

2 fj,iK 


i=2,3 

exp(As,i(Po)) 4- NT{to,nt). 


Combining the above estimate with (6.9), (6.23) and (6.36), we have for large enough n and nt > ei. 
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32 


P(7^(n,^)) < 


exp 


- l) 0 ?’^(A 2 ,ei) !> + 2 ^ exp<{ -( 


K 


i=2,3 




K 




+ 2 ^ exp|-(^i - 2 ei)^^-i^^ 02 ’*(/Xi,ei)| + 2 ^ exp |-(/Xint - l) 0 i’*(/i 2 , ei)| 


2=2,3 


+ 2 ^ (^r + ei)niexp | exp(A^,i 


2=2,3 


(Po)) 


+ X! -l)0i’*(/Xi,ei)|+iVr(to,nt) 

+ k” exp |-(/^ 2 nt - l)0i’^(/i2,1)} + exp |-(Aint - l)0“’^(Ai, 1)| 

+ exp | —(A 2 nt — 1)0“’^(A2, 1)|. (6.45) 

Thus, we can find positive constants ji, i = 1, 2, 3,4, such that for large enough n and nt> ei, 

P(72.(n, t)) < 7 i(nt + + js^nt + 1)^71“'’'^*“. 


□ 


6.3. Proof of Lemma 5 .7 


Recall the stopping times {r^lfeeNo defined in (6.5) and note from Lemma 4.4 that when t G [''■^fc_ 2 ) 


A-‘(*) A 

, Qm 

— y/n'^ 

Ml 

M2 


^ QUt) 

Qm 

_ ^ f 

- M? 

M^ 

M2 V 

> QUt) 

- M? 

Ms / 

M2M3 ' 



V 

Qm , Qm 


Ms 


Ms 


1 


= ^(l- 




M>2/ 


Qi{t) - 


Ms 

M 2 M 3 


■L" + — - 


1 

m1 


Ms 
M 2 M 3 
Ms 
M 2 M 3 


g-(t) 


(6.46) 


Thus for such t the inequality in (5.5) clearly holds (with a suitable choice of Ci). 

Next consider intervals of the form k,l G N, we define a sequence of {Gi(t)}t>o 

stopping times within [v^i- 2 ^ 021 - 1 ) ^ [''” 2 ) 0 - 1 ’'^ 2 fe) follows: For m G N, 


^0 

^2m-l 

^2m 


“ M2/’-2) 

= T 2 k A M 2/-1 A inf |t > C 2 ml '2 : W'^(t) - {W^{t)/y/n) < 50 } , 

= T 2 k A V 21-1 A inf [t > {W^(t)/y/n) > 50 } ■ 


Then 

for t e [C2m-2.C™-i). W”{t) - Vn4'(VF2”(0/Vn) > 9o 

and so for such t the inequality in (5.5) holds trivially. 


(6.47) 
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Also, from the definition of {?7"’^}ieNo and (6.12), we see that for t G [^ 2 /- 2 > 

n n 

Qm - Q?(t) < - L” + 2). 

Ml M 2 


33 


(6.48) 


Thus from Definition 3.2 we see that over this interval Server 1 does not work on Buffer 2. Thus letting 

A^{t) = A-{t) - A^(C”™"i\), S^{t) = s^{T-{t)) - 53”(r3"(C”™"i^)), 

for t G <32 (i) = Q 2 (C 2 ^-i) + ^2 (t) and Q^{t) = Q^(C 2 ml\) - Using the nonde¬ 

creasing and Lipschitz continuity property of 4/, we now have for t G [CJm-u 


A"(i) = ^ ^ 


> 


Mf M2 
02(C2”™"l'l)+^2(0 


Qm+Qiit) 


> 



M^ 

A'^it) 

QnC2A:^A 

M^ 

M? 

— ^/n 

fQ^iC2A^ 

A^it) 

QUC 2 AA) 

M^ 

M? 


- 


\/nM3 

( Q 2 (C2^^i\)+ m) +gg(C2^^i^) - sm ' 

\ \/nM3 


Q?(C2^^i\) ^ Q2{C2^'A) _ + Q 3 (C2”™":^i) 


Mf M 2 

Q 2 (C2r.^i) + m) + Q^(C2^"i'i) - sm' 




- ^ 


Vnfis J 

Q2(C2^^i\) + Qg(C2^^:^i) ^ 

\/nM3 


+ A"(c”;fi\)- 


M3 

M>2 


(i^(t)-4"(t)) + . 


Using the Lipschitz property of again we see that there exists Cq > 0 (depending only on 4' and the 
rate parameters) such that 


A"(C2”;fl\) > A"(C2^"y\-) - Co > go - Co, 


where the last inequality is from (6.47). Furthermore, for all t G C^m’*)> ^2(0 A Q^i^) and 

(from (6.48)) Qi(t) < ^(C" — L” -f 2). Thus we have for such t (in fact for all t G [M 2 (- 2 ’ M 2 /’-i))’ 


A”(t) > 


1 


M3 


/ i ll' 11 

M 3 M 2 


02 (i)-—(C”-L” + 2) + go-Co. 
M 2 


(6.49) 


We have therefore shown that (5.5) holds for all t G [M 2 Z- 2 ’ M 2 /-i)- 

We finally consider the case when t G M 2 z’^)(with a suitable choice of Ci, C 2 ). As noted below 

(6.16), over this interval Server 1 works on Buffer 1 continuously, and since, from the definition of 
over this interval Q^it) > L" — 1, Server 2 works on Buffer 3 continuously. Let 


Am = Am - ^ 2 (m2/^), sm = s^{t - r,-t, +- ssmiv^tA)- 
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Thus again from the monotonicity and Lipschitz continuity property of 'f', we have for t G 




Ml M2 

n,k 


Q^(t) + Qg(f) 

\/^M3 

n,k 


QUvT-i) + {Qnt)-QnvT-i)] , QB{V2{C)+Am 


— y/n'^ 


Ml M2 

'QUV2{-i) + m) + Qa (M2/^) - sm' 




QUt)-Qiiv2i-i) , 


Mf 


M2 


+ 


Qi(^2Ti) , Q2(M2Ti) 


Ml 


Ml 


— •v/n'f' 


'Q2(%/^)+Q3(M2”’^)' 




Q2 (m 2 ”’^)+ Am+Qmm) - ssit) 




— 


\ \/«M3 

Qm2i-i) + QUv2i-iy 


\/^M3 


q?w-w(^2/-i) , Am 


> + ^^iv2i-i) - -^Am - sm))^- 


Ml 


Ml 


MlM 2 


Also, from (6.49), 

A"(,7”’^)>A"(r72/^-)-t^o>- 
and using ( 6 . 12 ) 

'»n/ n,fe 


M3 


, ,n ! ,ri,, 

M 2 M 3 M 2 


Q?(M2r 1 ) < Qi (^ 2 ”-!-) +1 < 9r(^” - L" + 2) + 1 . 


Q 2 (%Ti-) - —(C-” - A” + 2) + 50 - 2Co, 
M 2 


Ml 


Ml 


Furthermore, for all t G [M 2 /- 1 ’ ^ 2 /^)) 


Am < Qm. and Q^(52i!i-) < q^(m2”’^) +1 < Qm +1- 


Using the above estimates, we have for t G [M21-1’ 


A^(t) > -(C" - L" + 2) - 1 - 

M 2 

>- —(C'"-L" + 2 )-l- 
M 2 


M3 


, ,n , .n-,, 

M 2 M 3 M 2 


1 


M3 


, ,n , ,n ,, 

M 2 M 3 M 2 

Thus for such t, (5.5) holds with suitable choice of Ci, C 2 as well. The result follows. 


g^(t)+A-(5”’^-)-Co 

(2Q"(t) + l)+5o-2Co. 


(6.50) 


□ 


Appendix A: One-dimensional Skorohod map 

We recall below the definition and basic properties of the 1-dimensional Skorohod map [25]. Recall 
Vi = {x G P([0,oo) : K.) : x(0) > 0}. 

Definition A.l (One-dimensional Skorohod Problem (SP)). Let x G Vi. A pair {z,y) G P([0,oo) : 
K+) X T’([0,00) : ]R_|_) is a solution of the Skorohod problem for x if the following hold. 

(i) For all t > 0, z{t) = x{t) + y{t) > 0. 

(ii) y satisfies the following: (a) 5(0) = 0, (b) y is nondecreasing, and (c) y increases only when z = 0, 

that is, z{t)dy{t) = 0. 
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The following proposition summarizes some well known properties of the 1-dimensional SP (see [10] 
for a proof). 

Proposition A.2. 

(i) Let X € Vi. Then there exists a unique solution {z,y) G X>([0,oo) : R+) x 2?([0, oo) : R+) of the 
SP for X, which is given as follows: 

y{t) = - inf (a:(s) A 0), z{t) = x{t) - inf (a;(s) A 0), f > 0. 

0<s<i 0<s<i 

We write z = r(x), and refer to the map F : —>■ X>([0, oo) : R+) as the Skorohod map. Let 

I: P([0,oo) : R) —>■ 2?([0, oo) : R) he the identity functional. Then y = {T —X){x). 

(ii) The Skorohod map T is Lipschitz continuous in the following sense: For allt>0 and xi,X 2 G T>i, 

sup |r(a;i)(s) - r(a: 2 )(s)| < 2 sup |a;i(s) - a; 2 (s)|. 

0<s<t 0<s<t 

(iii) Fix X G Vi. Let (z,y) G X>([0, oo) : R+) x 2?([0,oo) : R_|_) be such that 

(a) z{t) = x{t) + y{t) >0, t>0, 

(b) y is nondecreasing with y(0) = 0. 

Then z(t) > r(x)(t), t > 0. 
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